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O (57) Abstract: In order to enable a data signal, such as an audio or video signal, to be marked with multiple watermarks a set of 
^ different codes or keys is used within a given application field. The set of codes/keys is known to a watermark embedder (20) and a 
Q watermark detector (22). In each watermark embedding step, whether it is a first embedding step or a subsequent embedding step, 

a different key is used. The embedder (22) includes a data extractor (24) for determining which key or keys has or have been used 

previously. A data generator (26) then selects a new key fxom the set of known keys. 
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Embedding Multiple Watennaiks 



This invention relates to a method of embedding multiple watermarks into data 
signals, particularly, but not limited to, a method of embedding multiple watermarks into 
audio and/or video signals. 

5 

Embedding data into audio or video signals by means of watennaridng 
technology is well known and is described in, for instance. Van der Veen M, Braekers F., 
Haitsma J., K^alker T., Lemma A. and Oomen W., 2001, Robust, multi-functional and higji- 
quality audio watermarking technology: Convention pzper, presented at the 110* Conv^tion 

10 of the AES, May 12-15, Amsterdam, The Netherlands, and Swanson M,, TwefDk A. and 
Boney L., 1998, Robust audio watermarking using perceptual masking: Signal Processing, 
66, pp. 337-355, both of which are incorporated herein by reference- 
In Figure 1 there is shown a general block diagram for bofli a watermark 
embedder 10 and a watermark detector 12. The basic functionality of a watermark embedder 

15 10 is the combining of a signal X with a set of user data D to produce an output signal Y. The 
signal X is tihe audio or video signal of interest, whereas the user data D is the watermark. 
Often, auditory or visual perception models 14 are deployed to control the quality of the 
ou^ut signal Y. In the watermark detector 12, the set of user data D is extracted fiom the 
watermarked output signal Y with a data extractor 16. 

20 In some implementations of the approach described above problems may be 

introduced when multi-marking signal Y. e.g. embedding multiple watermarks, i.e. stacking 
different watermark signals. If for example signal Y, with a data set Do is watermarked again, 
or multi-marked, with data set Di, the detector 12 will not be able to retrieve either data set 
Do or data set Di, or both. Besides these three options, it may also be more difficult to 

25 retrieve both data sets Both watermarks Do and Di are said to have destructive interference. 



It is an object of the present invention to address the above mentioned 
disadvantage. . 
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According to a first aspect of the present mveation a mettiod of embedding a 
watomaik in a sigaal comprises: 

checking a signal to be watermarked for a two-part watermaik, a first part of 
which comprises a first identifier portion and a second part of which comprises a first 
5 information portion; 

on finding said two-part watermark, the method includes identifying the first 
identifier portion and selecting a different identifier portion firom a set of identifier portions 
and combining the different identifier portion with an information portion of the watermark 
to be embedded; and 

10 on finding no two-part watermark, the method includes selecting an identifier 

portion fiom the set of identifier portions and combining the identifier portion with the 
information portion of the watermatk to be embedded; 

the identifier and information portions are then combined to produce ttie 
watermark for embedding. 
1 S The information portion preferably includes a payload of the wateimaik, 

preferably having the infoimation or instructional content of the watermark. 

Preferably, the identifier portions are substantially orthogonal to one another, 
preferably there is substantially no destructive interference between identifier portions. 
Preferably, the identifier portions in the set of identifier portions are chosen to be 
20 orthogonal/non-interfering with each other. 

The identifier portions may be carriers, in which case the information portions 
are xised to modulate the identifier portions. 

The method may include checking for more than one two-part watermark. 
Preferably, the method is operable to embed multiple two-part watermarks. Preferably, the 
25 method includes the selection of an identifier portion different to all identifier portion(s) of 
(an)other detected two-part watermark(s). 

Preferably, the set of identifier portions is in the form of a list Preferably, the 
first unused identifier portion in the list is used for combination with the information portion 
of the watermark to be embedded. 
30 The waterma]k(s) may include a label portion, which label portion may 

indicate the next identifier portion to be used. 

The identifi^ portions may be derived &om a seed that indicates what the 
identifier portions are. 

The watermark may be embedded using perception model techniques. 
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The method may include emhedding multiple two-part watermaiks. 

According to a second aspect of the invention, a method of detecting a 
watemiaik in a signal compiises: 

checking a signal of interest for at least one two-part watermark, a first part of 
5 each watennaik comprising an identifier portion and at least one corresponding information 
portion; 

checking the or each identifier portion for correspondence with an identifier 
portion in a set of known identifier portions; 

extracting each identifier portion corresponding to a member of the set to give 
10 its corresponding information portion, to thereby allow use of the information portion. 

The method preferably includes the detection of one or more two-part 

watermaiks. 

According to a third aspect, a watermark embedder operable to perform the 
method of the first aspect 
1 5 The watermark embedder may incorporate an identifier portion extractor, 

operable to extract the identifier portion fit)m a two-part watermark. 

The watermark embedder is preferably operable to supply an extracted 
identifier portion to an identifi^ portion generator, the latter preferably being operable to 
generate an identifier portion substantially dijfferent to the extracted identifier portion. The 
20 identifier portion generator preferably selects an identifier fipom a predetermined hst. 

The watermark embedder preferably includes a data combiner, operable to 
combine the identifier portion of a two-part watermark with an information portion thereof. 

The watermark embedder preferably includes a signal/data combiner operable 
to combine the two-part watermark with a data signal to be watermarked 
25 According to a fourth aspect a watermark detector operable to perform the 

method of the second aspect 

The watermark detector preferably includes an identifier portion extractor as 
described in the third aspect 

The watermaric detector preferably includes an identifier portion/information 
30 portion splitter operable to separate an extracted identifier portion firom its corresponding 
information portion. 

The watermark detector preferably includes a data extractor, which is 
preferably operable to extract a two-part watermark finom a received signal. 
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According to a fifth aspect a recordable medium carrying data having a 
watermaik embedded according to the first aspect 

All of the features described herein can be combined with any of the above 
aspects, in any combination. 



For a better understanding of the invention and to show how the same may be 
brought into effect, ^ecific embodiments will now be described, by way of sample, with 
reference to the accompanying drawings, in which: 
10 Figure 1 is a schematic flow diagram of a prior art watermaric embedder and 

detector stmcture; 

Figure 2 is a schematic flow diagram of an embedder and detector structure for 
embedding and detecting multiple watermaiics; 

Figure 3 is a schematic diagram of a watermark embedder and detector 
15 stmcture for a structure that makes use of multiple carriers for data to be watermarked; and 

Figure 4 is a schematic representation of multiple watermarks embedded in tiie 
same signal and variation of the watermarks over time. 

In order to enable a data signal, such as an audio or video signal, to be marked 
with multiple watermarks a set of different codes or keys is used within a given application 
20 field. The set of codes/keys is known to a watermark embedder 20 and a watermaik detector 
22. In each watermark embedding step, \^ether it is a fibrst embedding step or a subsequent 
embedding step, a difTerent key is used. The embedder 22 includes a data extractor 24 for 
determining which key or keys has or have been used previously. A data generator 26 then 
selects a new key fix>m the set of known keys. 

25 



As shown in Figure 2 a signal X is to be combined with a total data set Dtot- 
The total data set Dtot that wiD be combined with the input signal X consists of two parts: a 
pubUc data set Dpuwic and private data set Dpnv. The pubUc data set is the net data that needs 
30 to be embedded into the ii^ut signal X. The private data set is generated by the watermaik 
embedder 20 in such a way that the total Dtot is orthogonal to a total data set which has been 
produced using a different private data set Dpnv., such as a different carrier. The carriers may 
be orthogonal noise sequences. Given that total data sets Dtot with different private data sets, 
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Dpriv have less or no destructive interference, the individual total data sets Dtot (or 
watennaiks) are much more easily detectable, especially when multi-marking is poresent 

It should be home in mind that the number of watermarks that can be added to 
a signal is in practice limited, because eventually, given a suf&ciently great number of 
5 watermarks, severe degradation of the signal X being watermarked will result Consequently, 
the number of different watermarks to be included is limited, which number is the same or 
less than the number of private data sets Dpnv provided. 

For a particular watermarking ^plication a sufGcient number of different 
private data sets Dpriv is defined and is stored in the embedd^ 20 and the detector 22. Prior to 

1 0 combining the total data set Dtot with the ii^ut signal X, the watermark embedder 20 

evaluates the iiq>ut signal X to detemiine what private data sets Dpriv have already been used. 
The input signal X is analysed with a private data extractor 24. On the basis of the results of 
the exammation, a new private data set D^v is generated &om the set stored in ttie private 
data generator 26 and is combined with the public data set DpubUc in a private/public data 

15 combiner 28. Thus, the signal Dtot is formed and is combined with the signal X in a 
signal/data combiner 30. 

A perception model 32 may optionally be used in conjxmction with the input 
signal X whilst the combination with Dtot is being performed. The use of a perception model 
32 is the same as is well known in the prior art. 

20 In the detector 22, a data spUtter 34 generates a public output data section 

Dpubiic fox every private data set Dpriv iu an incoming signal Y. The incoming signal Y is 
processed using a data extractor 36 which extracts the signal Dtot Also, a private data 
extractor 24, the same as fiiat in the embedder 20, extracts a private data set Dpriv, which is 
input into the private/public data sptitter 34 to enable a signal of the public data set Dpubiic to 

25 be ou^ut Consequently, the detector 22 produces a series of public output data sets DpubUc, 
assuming that a number of private data sets are detected. For fliis reason a mechanism is 
required to indicate the origin or destination of the pubUc data sets Dpuwic. One way of 
solving this is to allocate address fields in the public data set DpuWic- Some cryptographic 
and/or error detection mechanism may be applied on the public data set, but this is optional. 

30 Any malicious user of a watermark detector should not be able to trace the address fields. 
Therefore, payload, or address fields may be encrypted. Moreover payload edacity may be 
used to transmit EEC bits, which will enhance detection performance. 

A particular embodiment of the system more generally described above in 
relation to Figure 2 is shown in Figure 3. The watermaridng algorithm described in Robust, 
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Multi-functional and High Quality Audio Watermaddng Technology (above) can be used as 
a basis for the following embodiment The content of that document is mcorporated herein by 
reference. The following watermaiking algorithm may be used: 



5 Wi(k)=Ws(k)Xi(k), 

where j indicates the frame number, Xi(k) the spectral representation of the frame signal Xi(n) 
and Wi(k:), the resulting frequency domain watermark. 

This equation is used on an audio sigaal that is segmented into frames and 

10 transformed to the frequency domain. The magnitudes of the Fourier coefficients are slightly 
modified by utilising a shifted watennark Ws(k). 

In this algorithm the private data set Dpriv is r^resented by a carrier Q that 
will be modulated (by modulator 39) by the public data set DpubKc- The signal/data combiner 
30 shown in Figure 2 is also present in Figure 3. The signal/data combiner 30 consists of the 

1 5 Fourier transform, a multiplication, and an inverse Fourier transform. The perception model 
32 can be a transient detector that disables embedding of the watermark Dtot in the vicinity of 
a detected traBsient. In the Figure 3 embodiment the private data extractor 24 of Figure 2 is 
replaced by a carrier detector 38. The carrier detector 38 in both the embedder 20 and the 
detector 22 use correlation techniques to determine the presence of a particular carrier (which 

20 in this instance is the example of the private data set) without interpreting the embedded data, 
A demodulator 40 determines for what cyclic shift correlation is mayiTnal by deriving 
correlation for diflferent cyclic shifts. Based on this the payload is extracted. 

In Figure 4 it is shown how multiple private data sets or carriers are used to 
obtam non-destructive interfering watermarks. A plurality of embedders 20 indicated at A to 

25 C in Figure 4 are used Embedder A starts embedding its data a t=ti and proceeds until t=t3, 
as shown by the x-axis of the gr^h shown in Figure 4. Assuming that the original signal x is 
watennark free, the carrier detector 38 (or private data extractor 24) will not detect any 
carrier (private data set) and proposes to use the first carrier Q)rivate data set) available in the 
list 42 shown in Figure 3. In this case Dpnv = Co for i=0. 

30 A second watermark embedder B starts embedding its data at t==to and 

proceeds until t=t4. The carrier detector 38 (private data extractor 24) will not detect any 
carrier (private data set) in the period fiiom tF=to through to t=ti so it proposes to use the first 
carrier (private data set) available in the carrier list 42; Dpnv = Q) for i=0. From tr=ti through 
t==t3 carrier (private data set) Co will be detected as it is used by embedder A earlier. 
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Therefore, it is proposed to use the second carrier (private data set) available in the carrier list 
42; Dpriv = Ci for i=i . In the interval fiom t==t3 through to t==t4 no other carrier (private data 
set) will be detected and therefore embedder B will continue by using carrier (private data 
set) Co- The next watermark embedder C starts embedding its public data at tF=t2 and proceeds 

5 until 1=t5. With the same strategy as embedder B, the carrier*s (private data set) C2, Ci and Co 
are selected for the three time intervals, as shown in Figure 4. 

In the examples of Figures 3 and 4, the carrier (private data set) lists 42 are 
ordered in a way to allow determination of which private data sector was embedded first, 
second, third or later. The private data sector embedded with carrier Chi is embedded later 

10 than flie private data set embedded with the carrier Q. In the embedder 20 it is proposed to 
use the next carrier fiom the carrier list 42. In principal, it would be sufficient to select an 
unused carrier Scorn the Ust. Using an audit carrier list 42 may also be beneficial for the 
algorithmic complexity of the carrier detector 38. 

An optional way in which to reduce the algorithmic complexity of tiie private 

1 5 data set detector 24 or carrier detector 38 is as follows. N©ct to the total data set Dtot 

additional information concerning the use of private data sets may be embedded in Ihe signal. 
For example, it is possible to look ahead with the private data set extractor 24 in the 
embedder 20 and if some change in private data set usage is foreseen, this may be signalled 
explicitiy in the watermark. Consequently, subsequent private data set extractors 24, both 

20 embedder 20 and detector 22 can use this signal instead of or as a si^port in performing 
private data set extraction operations. In relation to the above, it is deemed to be 
advantageous to spend a few bits of information content in the payload to announce that the 
key is about to change, because this better enables the embedder 20 of detector 22 to account 
for tiiose changes. 

25 In Figure 3, the carrier detector 38 is used to extract the particular carrier Co to 

C3 that has been embedded by the embedder 20. The demodulator is then used to remove the 
carrier from the Dtot to reveal the public data set Do to D3. In this way, any one of the 
different public watermarks Do to D3 is extracted from the signal Y without any interference 
from any of the other watermarks and without any negative effect on the original signal X 

30 that has received the watermark A carrier is selected fix)m the list, and based on the carrier, 
either, DO, Dl or D3 are extracted. 

The method may include the private data sets being generated &om a seed for 
private data sets 
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In the mettiod of inserting and detecting multiple watermaiks disclosed above, 
significant advantages are derived because of the lack of interference, or minimised 
interference, between multiple watermaiks. This provides advantages in that tiie audio or 
video signal to be watermaiiced can be the result of an editing process, which may comprise a 
5 composition of midtiple original worics, some of which akeady have an embedded 

watennaik. In this case, the selected private data set may change in.time, as described above. 

The invention can be summarized as follows. In order to enable a data signal, 
such as an audio or video signal, to be marked with multiple watermarks a set of different 
codes or keys is used within a given application field. The set of codes/keys is known to a 
1 0 watemaaik embedder (20) and a watamaik detector (22). In each watermark embedding step, 
whether it is a first embedding step or a subsequent embedding step, a different key is used. 
The embedder (22) includes a data extractor (24) for detennining which key or keys has or 
have been used previously. A data generator (26) then selects a new key bom the set of 
known keys, 

15 



^1 
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CLAIMS: 



1 . A method of embedding a watermark in a signal comprises: 

checking a signal to be watermarked for a two-part watermark, a jBrst part of 
which comprises a first identifier portion and a second part of which comprises a first 
information portion; 

5 on finding said two-part watermark, the method includes identifying flie first 

identifier portion and selecting a different identifier portion fix>m a set of identifiCT portions 
and combining the different identifier portion witii an information portion of the watermark 
to be embedded; and 

on finding no two-part watermark, the method includes selecting an identifier 
10 portion fiom the set of identifier portions and combining the identifier portion with the 
information portion of the watermark to be embedded; 

the identifier and information portions are then combined to produce the 
watermark for embedding. 

15 2. A method as claimed in claim 1 , in which the mformation portion includes a 

payload of the watermark, having information or instructional content of tiie watermark. 

3. A method as claimed in either claim 1 or claim 2, in which the identifier 
portions are substantially orthogonal to one another. 

20 

4. A method as claimed in any preceding claim, in which the identifier portions 
in the set of identifier portions are chosen to be orthogonal/non-interfering with each otibier. 

5. A method as claimed in any preceding claim, which includes checking for 
25 more than one two-part watermark. 

6. A method as claimed in any preceding claim, which is operable to embed 
multiple two-part watermarks. 



/ 
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7. A method as claimed in any preceding claim, in wbich the set of identifier 

portions is in the form of a list, the first miused identifier portion in the list being used for 
combination with the information portion of the watermark to be embedded. 

5 8. A method as claimed in claim 7, in which the watermark includes a label 

portion, \^4uch indicates the next identifier portion that should be u^ed. 

9. A method as claimed in any preceding claim, in which the identifier portions 
are carriers, and the information portions are used to modulate the identifier portions. 

10 

10. A method of detecting a watermark in a signal comprises: 

checldng a signal of interest for at least one two-part watermark, a first part of 
each watermark comprising an identifier portion and at least one corresponding information 
portion; 

1 5 checking flie or each identifier portion for correspondence with an identifier 

portion in a set of known identifier portions; 

extracting each identifier portion corresponding to a member of the set to give 
its corresponding information portion, to thereby allow use of the information portion. 

20 1 1 . A watermark embedder operable to perform the method of any one of claims 1 

to 9. 

12. A watermark detector operable to perform the method of claim 1 0. 



25 



13. 

the method 



A recordable medium carrying data having a watermark embedded accorded to 
of any one of claims 1 to 9. 
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